{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uK0lOwXqCcci"
      },
      "source": [
        "This notebook shows how to\n",
        "\n",
        "\n",
        "1.   Set up & configure Promptfoo\n",
        "2.   Set up secrets (such as an OpenAI or Anthropic API key)\n",
        "3.   Run an eval of LLM models, prompts and outputs\n",
        "4.   Run another eval that includes Python code in this notebook\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5yj5q5FMSn6v"
      },
      "source": [
        "# Node/Promptfoo setup"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qBI5MsRN-mw9"
      },
      "source": [
        "Start by installing the Node.js dependency."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CLsWqecMRPNn"
      },
      "outputs": [],
      "source": [
        "# Node version 16 or higher is required for PromptFoo to work.\n",
        "# The current version of Node in Google Colab is 14.16.\n",
        "# Upgrade to the latest version of Node using the n package manager. Learn more at https://nodejs.org/en/download/package-manager#n\n",
        "!npm install -g n\n",
        "!n latest"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n61ejK4u-tFy"
      },
      "source": [
        "Next, we'll install and initialize promptfoo."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WNzZutVjRab-",
        "outputId": "547a1c03-a9ed-484c-ecf7-38ddb101ef21"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "env: npm_config_yes=true\n",
            "\u001b[32m\u001b[1mWrote prompts.txt and promptfooconfig.yaml. Open README.md to get started!\u001b[22m\u001b[39m\n"
          ]
        }
      ],
      "source": [
        "# Set up promptfoo\n",
        "%env npm_config_yes=true\n",
        "!npx promptfoo@latest init"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BWbwGrFLSzzQ"
      },
      "source": [
        "# Configure promptfoo"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4WtI9tXZ98sB"
      },
      "source": [
        "First, we set up the prompts. See https://promptfoo.dev/docs/configuration/parameters for more info on prompt files."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8bgtMk0n86yj",
        "outputId": "ce1392af-5271-4d97-9832-c912f3588e3e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overwriting prompts.txt\n"
          ]
        }
      ],
      "source": [
        "%%writefile prompts.txt\n",
        "You're an ecommerce chat assistant for a shoe company.\n",
        "Answer this user's question: {{name}}: \"{{question}}\"\n",
        "---\n",
        "You're a smart, bubbly chat assistant for a shoe company.\n",
        "Answer this user's question: {{name}}: \"{{question}}\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T_zcpupd14Rm"
      },
      "source": [
        "Next, we set up the configuration. See https://promptfoo.dev/docs/configuration/guide for more info on configuration."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EUst0Tmv9Pfy",
        "outputId": "533b38c5-af6a-4265-a3e9-dd2c7f0a5e04"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overwriting promptfooconfig.yaml\n"
          ]
        }
      ],
      "source": [
        "%%writefile promptfooconfig.yaml\n",
        "prompts: [prompts.txt]\n",
        "providers: [openai:chat:gpt-3.5-turbo-0613, openai:chat:gpt-4]\n",
        "tests:\n",
        "  - vars:\n",
        "      name: Bob\n",
        "      question: Can you help me find a pair of sandals on your website?\n",
        "  - vars:\n",
        "      name: Jane\n",
        "      question: Do you have any discounts available?\n",
        "  - vars:\n",
        "      name: Dave\n",
        "      question: What are your shipping and return policies?\n",
        "  - vars:\n",
        "      name: Jim\n",
        "      question: Can you provide more info on your hiking boot options?\n",
        "  - vars:\n",
        "      name: Alice\n",
        "      question: What is the latest trend in winter footwear?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5K21inptxl73"
      },
      "source": [
        "# Set up secrets"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2u7gsi86xqoC"
      },
      "source": [
        "This section loads a `secrets.json` file from your Google Drive.  This is only necessary if you are using paid APIs like OpenAI or Anthropic.\n",
        "\n",
        "The file should look something like this:\n",
        "\n",
        "```\n",
        "{\n",
        "  OPENAI_API_KEY=\"sk-abc123\"\n",
        "}\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OBPVTKY_x0kY",
        "outputId": "ae01f547-641c-4724-f762-d0daad678abf"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"
          ]
        }
      ],
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "jYr-v2_Cx-BL"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import json\n",
        "\n",
        "with open('/content/drive/MyDrive/Projects/promptfoo/secrets.json') as f:\n",
        "  secrets = json.load(f)\n",
        "\n",
        "for key, value in secrets.items():\n",
        "    os.environ[key] = value"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yDgQDQiKvKCN"
      },
      "source": [
        "# Run the eval"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cdJXjemM1UDK"
      },
      "source": [
        "First, run the eval - this will produce a quick side-by-side table view."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d5hZGY2svL6D",
        "outputId": "756f850d-0641-4a58-f551-03b6c9d33248"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Creating cache folder at /root/.promptfoo/cache.\n",
            "\n",
            "\u001b[90m┌────────────────────\u001b[39m\u001b[90m┬────────────────────\u001b[39m\u001b[90m┬────────────────────\u001b[39m\u001b[90m┬────────────────────\u001b[39m\u001b[90m┬────────────────────\u001b[39m\u001b[90m┬────────────────────┐\u001b[39m\n",
            "\u001b[90m│\u001b[39m\u001b[1m\u001b[34m name               \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m question           \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m [openai:gpt-3.5-tu \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m [openai:gpt-4] You \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m [openai:gpt-3.5-tu \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m [openai:gpt-4] You \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m rbo-0613] You're a \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m 're an ecommerce c \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m rbo-0613] You're a \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m 're a smart, bubbl \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m n ecommerce chat a \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m hat assistant for  \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m  smart, bubbly cha \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m y chat assistant f \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m ssistant for a sho \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m a shoe company.    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m t assistant for a  \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m or a shoe company. \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m e company.         \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m Answer this user's \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m shoe company.      \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m Answer this user's \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m Answer this user's \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m  question: {{name} \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m Answer this user's \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m  question: {{name} \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m  question: {{name} \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m }: \"{{question}}\"  \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m  question: {{name} \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m }: \"{{question}}\"  \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m }: \"{{question}}\"  \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m }: \"{{question}}\"  \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                    \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\n",
            "\u001b[90m├────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────┤\u001b[39m\n",
            "\u001b[90m│\u001b[39m Bob                \u001b[90m│\u001b[39m Can you help me fi \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m nd a pair of sanda \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mAssistant:\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mOf course,\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mOf course,\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mAbsolutely\u001b[22m      \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m ls on your website \u001b[90m│\u001b[39m \u001b[1m Of course, Bob! I\u001b[22m \u001b[90m│\u001b[39m \u001b[1m Bob! I'd be happy\u001b[22m \u001b[90m│\u001b[39m \u001b[1m Bob! I'd be delig\u001b[22m \u001b[90m│\u001b[39m \u001b[1m, Bob! I'd be happ\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m ?                  \u001b[90m│\u001b[39m \u001b[1m'd be happy to hel\u001b[22m \u001b[90m│\u001b[39m \u001b[1m to assist you. To\u001b[22m \u001b[90m│\u001b[39m \u001b[1mhted to help you f\u001b[22m \u001b[90m│\u001b[39m \u001b[1my to assist you. T\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mp you find a pair \u001b[22m \u001b[90m│\u001b[39m \u001b[1m help you better, \u001b[22m \u001b[90m│\u001b[39m \u001b[1mind a pair of sand\u001b[22m \u001b[90m│\u001b[39m \u001b[1mo help you better,\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mof sandals on our \u001b[22m \u001b[90m│\u001b[39m \u001b[1mcould you please p\u001b[22m \u001b[90m│\u001b[39m \u001b[1mals on our website\u001b[22m \u001b[90m│\u001b[39m \u001b[1m could you please \u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mwebsite. Could you\u001b[22m \u001b[90m│\u001b[39m \u001b[1mrovide more detail\u001b[22m \u001b[90m│\u001b[39m \u001b[1m. Could you please\u001b[22m \u001b[90m│\u001b[39m \u001b[1mprovide me with mo\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1m please let me kno\u001b[22m \u001b[90m│\u001b[39m \u001b[1ms? Are you looking\u001b[22m \u001b[90m│\u001b[39m \u001b[1m let me know what \u001b[22m \u001b[90m│\u001b[39m \u001b[1mre details? For in\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mw what specific ty\u001b[22m \u001b[90m│\u001b[39m \u001b[1m for a specific st\u001b[22m \u001b[90m│\u001b[39m \u001b[1mspecific type of s\u001b[22m \u001b[90m│\u001b[39m \u001b[1mstance, are you lo\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mpe of sandals you \u001b[22m \u001b[90m│\u001b[39m \u001b[1myle, color, or siz\u001b[22m \u001b[90m│\u001b[39m \u001b[1mandals you are loo\u001b[22m \u001b[90m│\u001b[39m \u001b[1moking for a specif\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mare looking for?\u001b[22m   \u001b[90m│\u001b[39m \u001b[1me?\u001b[22m                 \u001b[90m│\u001b[39m \u001b[1mking for? Are you \u001b[22m \u001b[90m│\u001b[39m \u001b[1mic style, color, o\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1minterested in men'\u001b[22m \u001b[90m│\u001b[39m \u001b[1mr size?\u001b[22m            \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1ms or women's sanda\u001b[22m \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mls? And do you hav\u001b[22m \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1me any particular s\u001b[22m \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mtyle, ...\u001b[22m          \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m├────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────┤\u001b[39m\n",
            "\u001b[90m│\u001b[39m Jane               \u001b[90m│\u001b[39m Do you have any di \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m scounts available? \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mAssistant:\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mYes, Jane,\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mAssistant:\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mAbsolutely\u001b[22m      \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1m Hello Jane! Thank\u001b[22m \u001b[90m│\u001b[39m \u001b[1m we do have discou\u001b[22m \u001b[90m│\u001b[39m \u001b[1m Hi Jane! Thank yo\u001b[22m \u001b[90m│\u001b[39m \u001b[1m, Jane! We often h\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1m you for reaching \u001b[22m \u001b[90m│\u001b[39m \u001b[1mnts available from\u001b[22m \u001b[90m│\u001b[39m \u001b[1mu for reaching out\u001b[22m \u001b[90m│\u001b[39m \u001b[1mave various discou\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mout. Yes, we do ha\u001b[22m \u001b[90m│\u001b[39m \u001b[1m time to time. Cur\u001b[22m \u001b[90m│\u001b[39m \u001b[1m to us. Yes, we do\u001b[22m \u001b[90m│\u001b[39m \u001b[1mnts and promotiona\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mve discounts avail\u001b[22m \u001b[90m│\u001b[39m \u001b[1mrently, we are off\u001b[22m \u001b[90m│\u001b[39m \u001b[1m have some excitin\u001b[22m \u001b[90m│\u001b[39m \u001b[1ml offers available\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mable. Could you pl\u001b[22m \u001b[90m│\u001b[39m \u001b[1mering a 10% discou\u001b[22m \u001b[90m│\u001b[39m \u001b[1mg discounts availa\u001b[22m \u001b[90m│\u001b[39m \u001b[1m. The specifics ca\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mease let me know w\u001b[22m \u001b[90m│\u001b[39m \u001b[1mnt on your first o\u001b[22m \u001b[90m│\u001b[39m \u001b[1mble at the moment.\u001b[22m \u001b[90m│\u001b[39m \u001b[1mn vary, so I recom\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mhich specific shoe\u001b[22m \u001b[90m│\u001b[39m \u001b[1mrder when you sign\u001b[22m \u001b[90m│\u001b[39m \u001b[1m Could you please \u001b[22m \u001b[90m│\u001b[39m \u001b[1mmend checking our \u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1ms you are interest\u001b[22m \u001b[90m│\u001b[39m \u001b[1m up for our newsle\u001b[22m \u001b[90m│\u001b[39m \u001b[1mlet me know which \u001b[22m \u001b[90m│\u001b[39m \u001b[1mwebsite or subscri\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1med in? This way, I\u001b[22m \u001b[90m│\u001b[39m \u001b[1mtter. Also, we hav\u001b[22m \u001b[90m│\u001b[39m \u001b[1mspecific shoes you\u001b[22m \u001b[90m│\u001b[39m \u001b[1mbing to our newsle\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1m can provide you w\u001b[22m \u001b[90m│\u001b[39m \u001b[1me a sale section o\u001b[22m \u001b[90m│\u001b[39m \u001b[1m are interested in\u001b[22m \u001b[90m│\u001b[39m \u001b[1mtter for the most \u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mith the most accur\u001b[22m \u001b[90m│\u001b[39m \u001b[1mn our website wher\u001b[22m \u001b[90m│\u001b[39m \u001b[1m? That way, I can \u001b[22m \u001b[90m│\u001b[39m \u001b[1mup-to-date informa\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mate information re\u001b[22m \u001b[90m│\u001b[39m \u001b[1me you can find sho\u001b[22m \u001b[90m│\u001b[39m \u001b[1mprovide you with t\u001b[22m \u001b[90m│\u001b[39m \u001b[1mtion. You can also\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mgarding any availa\u001b[22m \u001b[90m│\u001b[39m \u001b[1mes at discounted p\u001b[22m \u001b[90m│\u001b[39m \u001b[1mhe most accurate i\u001b[22m \u001b[90m│\u001b[39m \u001b[1m check our social \u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mble di...\u001b[22m          \u001b[90m│\u001b[39m \u001b[1mrices....\u001b[22m          \u001b[90m│\u001b[39m \u001b[1mnforma...\u001b[22m          \u001b[90m│\u001b[39m \u001b[1mmedia ...\u001b[22m          \u001b[90m│\u001b[39m\n",
            "\u001b[90m├────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────┤\u001b[39m\n",
            "\u001b[90m│\u001b[39m Dave               \u001b[90m│\u001b[39m What are your ship \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m ping and return po \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mAssistant:\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mWe offer f\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mAssistant:\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mOur shippi\u001b[22m      \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m licies?            \u001b[90m│\u001b[39m \u001b[1m Hello Dave! Thank\u001b[22m \u001b[90m│\u001b[39m \u001b[1mree standard shipp\u001b[22m \u001b[90m│\u001b[39m \u001b[1m Hello Dave! Thank\u001b[22m \u001b[90m│\u001b[39m \u001b[1mng policy is as fo\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1m you for your inte\u001b[22m \u001b[90m│\u001b[39m \u001b[1ming on all orders \u001b[22m \u001b[90m│\u001b[39m \u001b[1m you for your inte\u001b[22m \u001b[90m│\u001b[39m \u001b[1mllows: We offer fr\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mrest in our shoe c\u001b[22m \u001b[90m│\u001b[39m \u001b[1mover $50. For orde\u001b[22m \u001b[90m│\u001b[39m \u001b[1mrest in our shoe c\u001b[22m \u001b[90m│\u001b[39m \u001b[1mee standard shippi\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mompany. Our shippi\u001b[22m \u001b[90m│\u001b[39m \u001b[1mrs under $50, the \u001b[22m \u001b[90m│\u001b[39m \u001b[1mompany. I'd be hap\u001b[22m \u001b[90m│\u001b[39m \u001b[1mng on all orders o\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mng policy is as fo\u001b[22m \u001b[90m│\u001b[39m \u001b[1mstandard shipping \u001b[22m \u001b[90m│\u001b[39m \u001b[1mpy to provide you \u001b[22m \u001b[90m│\u001b[39m \u001b[1mver $50. For order\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mllows: We offer fr\u001b[22m \u001b[90m│\u001b[39m \u001b[1mfee is $5.99. Orde\u001b[22m \u001b[90m│\u001b[39m \u001b[1mwith information a\u001b[22m \u001b[90m│\u001b[39m \u001b[1ms under $50, a fla\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mee standard shippi\u001b[22m \u001b[90m│\u001b[39m \u001b[1mrs are typically p\u001b[22m \u001b[90m│\u001b[39m \u001b[1mbout our shipping \u001b[22m \u001b[90m│\u001b[39m \u001b[1mt rate shipping fe\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mng on all orders w\u001b[22m \u001b[90m│\u001b[39m \u001b[1mrocessed within 1-\u001b[22m \u001b[90m│\u001b[39m \u001b[1mand return policie\u001b[22m \u001b[90m│\u001b[39m \u001b[1me of $5.99 applies\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mithin the United S\u001b[22m \u001b[90m│\u001b[39m \u001b[1m2 business days an\u001b[22m \u001b[90m│\u001b[39m \u001b[1ms.\u001b[22m                 \u001b[90m│\u001b[39m \u001b[1m. Orders are typic\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mtates. Once your o\u001b[22m \u001b[90m│\u001b[39m \u001b[1md standard shippin\u001b[22m \u001b[90m│\u001b[39m \u001b[1mFor shipping, we o\u001b[22m \u001b[90m│\u001b[39m \u001b[1mally processed wit\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mrder is placed, it\u001b[22m \u001b[90m│\u001b[39m \u001b[1mg usually takes 5-\u001b[22m \u001b[90m│\u001b[39m \u001b[1mffer free standard\u001b[22m \u001b[90m│\u001b[39m \u001b[1mhin 1-2 business d\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1m typically takes 1\u001b[22m \u001b[90m│\u001b[39m \u001b[1m7 business days.\u001b[22m   \u001b[90m│\u001b[39m \u001b[1m shipping on all o\u001b[22m \u001b[90m│\u001b[39m \u001b[1mays and standard s\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1m-3 business days f\u001b[22m \u001b[90m│\u001b[39m \u001b[1mAs for returns, we\u001b[22m \u001b[90m│\u001b[39m \u001b[1mrders within the c\u001b[22m \u001b[90m│\u001b[39m \u001b[1mhipping usually ta\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mor pro...\u001b[22m          \u001b[90m│\u001b[39m \u001b[1m accep...\u001b[22m          \u001b[90m│\u001b[39m \u001b[1mountry. Once your \u001b[22m \u001b[90m│\u001b[39m \u001b[1mkes 5-...\u001b[22m          \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mor...\u001b[22m              \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m├────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────┤\u001b[39m\n",
            "\u001b[90m│\u001b[39m Jim                \u001b[90m│\u001b[39m Can you provide mo \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m re info on your hi \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mAssistant:\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mAbsolutely\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mOf course,\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mAbsolutely\u001b[22m      \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m king boot options? \u001b[90m│\u001b[39m \u001b[1m Of course, Jim! W\u001b[22m \u001b[90m│\u001b[39m \u001b[1m, Jim! We offer a \u001b[22m \u001b[90m│\u001b[39m \u001b[1m Jim! I'd be happy\u001b[22m \u001b[90m│\u001b[39m \u001b[1m, Jim! We offer a \u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1me offer a wide ran\u001b[22m \u001b[90m│\u001b[39m \u001b[1mvariety of hiking \u001b[22m \u001b[90m│\u001b[39m \u001b[1m to provide you wi\u001b[22m \u001b[90m│\u001b[39m \u001b[1mvariety of hiking \u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mge of hiking boot \u001b[22m \u001b[90m│\u001b[39m \u001b[1mboots designed for\u001b[22m \u001b[90m│\u001b[39m \u001b[1mth more informatio\u001b[22m \u001b[90m│\u001b[39m \u001b[1mboots designed to \u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1moptions to suit di\u001b[22m \u001b[90m│\u001b[39m \u001b[1m different terrain\u001b[22m \u001b[90m│\u001b[39m \u001b[1mn on our hiking bo\u001b[22m \u001b[90m│\u001b[39m \u001b[1mprovide comfort, d\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mfferent needs and \u001b[22m \u001b[90m│\u001b[39m \u001b[1ms and weather cond\u001b[22m \u001b[90m│\u001b[39m \u001b[1mot options. We hav\u001b[22m \u001b[90m│\u001b[39m \u001b[1murability, and sup\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mpreferences. Our h\u001b[22m \u001b[90m│\u001b[39m \u001b[1mitions. \u001b[22m           \u001b[90m│\u001b[39m \u001b[1me a wide range of \u001b[22m \u001b[90m│\u001b[39m \u001b[1mport for all your \u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1miking boots are de\u001b[22m \u001b[90m│\u001b[39m \u001b[1m1. **Trail Hiking \u001b[22m \u001b[90m│\u001b[39m \u001b[1mhiking boots desig\u001b[22m \u001b[90m│\u001b[39m \u001b[1moutdoor adventures\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1msigned with durabi\u001b[22m \u001b[90m│\u001b[39m \u001b[1mBoots**: These are\u001b[22m \u001b[90m│\u001b[39m \u001b[1mned to cater to di\u001b[22m \u001b[90m│\u001b[39m \u001b[1m. Here are some of\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mlity, comfort, and\u001b[22m \u001b[90m│\u001b[39m \u001b[1m designed for mudd\u001b[22m \u001b[90m│\u001b[39m \u001b[1mfferent needs and \u001b[22m \u001b[90m│\u001b[39m \u001b[1m our popular optio\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1m performance in mi\u001b[22m \u001b[90m│\u001b[39m \u001b[1my or rocky trails.\u001b[22m \u001b[90m│\u001b[39m \u001b[1mpreferences. Here \u001b[22m \u001b[90m│\u001b[39m \u001b[1mns:\u001b[22m                \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mnd. Here are some \u001b[22m \u001b[90m│\u001b[39m \u001b[1m They are waterpro\u001b[22m \u001b[90m│\u001b[39m \u001b[1mare a few of our p\u001b[22m \u001b[90m│\u001b[39m \u001b[1m1. **Trail Master \u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mkey features and o\u001b[22m \u001b[90m│\u001b[39m \u001b[1mof, durable, and p\u001b[22m \u001b[90m│\u001b[39m \u001b[1mopular options:\u001b[22m    \u001b[90m│\u001b[39m \u001b[1m3000**: This boot \u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mptions you can con\u001b[22m \u001b[90m│\u001b[39m \u001b[1mrovide excellent t\u001b[22m \u001b[90m│\u001b[39m \u001b[1m1. Trailblazer Hik\u001b[22m \u001b[90m│\u001b[39m \u001b[1mis waterproof and \u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1msider:...\u001b[22m          \u001b[90m│\u001b[39m \u001b[1mraction.\u001b[22m           \u001b[90m│\u001b[39m \u001b[1ming Boo...\u001b[22m         \u001b[90m│\u001b[39m \u001b[1mfeatures a rubber \u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1m2. *...\u001b[22m            \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1ms...\u001b[22m               \u001b[90m│\u001b[39m\n",
            "\u001b[90m├────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────\u001b[39m\u001b[90m┼────────────────────┤\u001b[39m\n",
            "\u001b[90m│\u001b[39m Alice              \u001b[90m│\u001b[39m What is the latest \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[\u001b[22m\u001b[39m           \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m  trend in winter f \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mAs an ecom\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mThe latest\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mHello Alic\u001b[22m      \u001b[90m│\u001b[39m \u001b[32m\u001b[1m22m\u001b[39mThe latest\u001b[22m      \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m ootwear?           \u001b[90m│\u001b[39m \u001b[1mmerce chat assista\u001b[22m \u001b[90m│\u001b[39m \u001b[1m trends in winter \u001b[22m \u001b[90m│\u001b[39m \u001b[1me! As a smart and \u001b[22m \u001b[90m│\u001b[39m \u001b[1m trends in winter \u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mnt for a shoe comp\u001b[22m \u001b[90m│\u001b[39m \u001b[1mfootwear include c\u001b[22m \u001b[90m│\u001b[39m \u001b[1mbubbly chat assist\u001b[22m \u001b[90m│\u001b[39m \u001b[1mfootwear include c\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1many, I can tell yo\u001b[22m \u001b[90m│\u001b[39m \u001b[1mhunky boots, espec\u001b[22m \u001b[90m│\u001b[39m \u001b[1mant for a shoe com\u001b[22m \u001b[90m│\u001b[39m \u001b[1mhunky boots, espec\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mu that the latest \u001b[22m \u001b[90m│\u001b[39m \u001b[1mially those with t\u001b[22m \u001b[90m│\u001b[39m \u001b[1mpany, I'm here to \u001b[22m \u001b[90m│\u001b[39m \u001b[1mially those with l\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mtrend in winter fo\u001b[22m \u001b[90m│\u001b[39m \u001b[1mrack soles for ext\u001b[22m \u001b[90m│\u001b[39m \u001b[1mhelp you out. The \u001b[22m \u001b[90m│\u001b[39m \u001b[1mug soles for extra\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1motwear is the rise\u001b[22m \u001b[90m│\u001b[39m \u001b[1mra grip and a fash\u001b[22m \u001b[90m│\u001b[39m \u001b[1mlatest trend in wi\u001b[22m \u001b[90m│\u001b[39m \u001b[1m grip and a fashio\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1m of chunky boots. \u001b[22m \u001b[90m│\u001b[39m \u001b[1mionable edge. Fur-\u001b[22m \u001b[90m│\u001b[39m \u001b[1mnter footwear is a\u001b[22m \u001b[90m│\u001b[39m \u001b[1mnable edge. Combat\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mChunky boots, with\u001b[22m \u001b[90m│\u001b[39m \u001b[1mlined boots and th\u001b[22m \u001b[90m│\u001b[39m \u001b[1mll about combining\u001b[22m \u001b[90m│\u001b[39m \u001b[1m boots are also ve\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1m their thick soles\u001b[22m \u001b[90m│\u001b[39m \u001b[1mose with shearling\u001b[22m \u001b[90m│\u001b[39m \u001b[1m style and functio\u001b[22m \u001b[90m│\u001b[39m \u001b[1mry popular. For a \u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1m and sturdy constr\u001b[22m \u001b[90m│\u001b[39m \u001b[1m details are also \u001b[22m \u001b[90m│\u001b[39m \u001b[1mnality. One popula\u001b[22m \u001b[90m│\u001b[39m \u001b[1mmore elegant look,\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1muction, are not on\u001b[22m \u001b[90m│\u001b[39m \u001b[1mpopular for their \u001b[22m \u001b[90m│\u001b[39m \u001b[1mr trend this seaso\u001b[22m \u001b[90m│\u001b[39m \u001b[1m knee-high and ove\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mly fashionable but\u001b[22m \u001b[90m│\u001b[39m \u001b[1mwarmth and style. \u001b[22m \u001b[90m│\u001b[39m \u001b[1mn is chunky boots \u001b[22m \u001b[90m│\u001b[39m \u001b[1mr-the-knee boots a\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1m also practical fo\u001b[22m \u001b[90m│\u001b[39m \u001b[1mOver-the-knee boot\u001b[22m \u001b[90m│\u001b[39m \u001b[1mwith lug soles. Th\u001b[22m \u001b[90m│\u001b[39m \u001b[1mre trending, parti\u001b[22m \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m                    \u001b[90m│\u001b[39m \u001b[1mr the ...\u001b[22m          \u001b[90m│\u001b[39m \u001b[1ms and ...\u001b[22m          \u001b[90m│\u001b[39m \u001b[1mese bo...\u001b[22m          \u001b[90m│\u001b[39m \u001b[1mcularl...\u001b[22m          \u001b[90m│\u001b[39m\n",
            "\u001b[90m└────────────────────\u001b[39m\u001b[90m┴────────────────────\u001b[39m\u001b[90m┴────────────────────\u001b[39m\u001b[90m┴────────────────────\u001b[39m\u001b[90m┴────────────────────\u001b[39m\u001b[90m┴────────────────────┘\u001b[39m\n",
            "======================================================================\n",
            "\u001b[32m✔\u001b[39m Evaluation complete.\n",
            "\n",
            "Run \u001b[92mpromptfoo view\u001b[39m to use the local web viewer\n",
            "Run \u001b[92mpromptfoo share\u001b[39m to create a shareable URL\n",
            "======================================================================\n",
            "\u001b[32m\u001b[1mSuccesses: 20\u001b[22m\u001b[39m\n",
            "\u001b[31m\u001b[1mFailures: 0\u001b[22m\u001b[39m\n",
            "Token usage: Total 3511, Prompt 762, Completion 2749, Cached 0\n",
            "Done.\n"
          ]
        }
      ],
      "source": [
        "!npx promptfoo@latest eval -c /content/promptfooconfig.yaml --no-progress-bar"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OBU2upLn1b0x"
      },
      "source": [
        "Now we can share and view this in the web viewer, which is easier to use and contains tools for drilling down and grading the outputs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QQq6hV6006Cv",
        "outputId": "93dd5b45-acce-4051-8d19-185469a67bab"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "View results: \u001b[92m\u001b[1mhttps://app.promptfoo.dev/eval/f:ffaf05af-6d34-4ff3-b035-2a73e0c97375\u001b[22m\u001b[39m\n"
          ]
        }
      ],
      "source": [
        "!npx promptfoo@latest share --yes"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NB4Xw1mh2ILr"
      },
      "source": [
        "# Eval a Python script in Colab"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v8o97GM_2LWo"
      },
      "source": [
        "This next section provides an example of how to run an eval on Python code within Colab.\n",
        "\n",
        "The goal in this example is to set up a LangChain implementation and compare it against vanilla GPT-4.  We're using LangChain in this example, but this approach works for any code."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1MIC2FYu2aV1"
      },
      "outputs": [],
      "source": [
        "!pip install langchain openai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RzTNvbVL-StX"
      },
      "source": [
        "Here's a chain that uses `LLMMathChain` to do math:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2O1JriWH2oX1",
        "outputId": "964857d5-cb79-4f59-a909-548dc7474c00"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overwriting langchain_example.py\n"
          ]
        }
      ],
      "source": [
        "%%writefile langchain_example.py\n",
        "import sys\n",
        "import os\n",
        "\n",
        "from langchain import OpenAI\n",
        "from langchain.chains import LLMMathChain\n",
        "\n",
        "llm = OpenAI(\n",
        "    temperature=0,\n",
        "    openai_api_key=os.getenv('OPENAI_API_KEY')\n",
        ")\n",
        "\n",
        "llm_math = LLMMathChain(llm=llm, verbose=True)\n",
        "\n",
        "llm_math.run(sys.argv[1])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rRyGlgld-a0C"
      },
      "source": [
        "Next, we'll set up the test cases."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1SSUsw-X2yZn",
        "outputId": "7c6d1cc9-7fd3-4076-f709-3cfd50601750"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overwriting mathconfig.yaml\n"
          ]
        }
      ],
      "source": [
        "%%writefile mathconfig.yaml\n",
        "prompts: mathprompt.txt\n",
        "providers:\n",
        "  - openai:chat:gpt-4-0613\n",
        "  - exec:python langchain_example.py\n",
        "tests:\n",
        "  - vars:\n",
        "      question: What is the cube root of 389017?\n",
        "  - vars:\n",
        "      question: If you have 101101 in binary, what number does it represent in base 10?\n",
        "  - vars:\n",
        "      question: What is the natural logarithm (ln) of 89234?\n",
        "  - vars:\n",
        "      question: If a geometric series has a first term of 3125 and a common ratio of 0.008, what is the sum of the first 20 terms?\n",
        "  - vars:\n",
        "      question: A number in base 7 is 3526. What is this number in base 10?\n",
        "  - vars:\n",
        "      question: If a complex number is represented as 3 + 4i, what is its magnitude?\n",
        "  - vars:\n",
        "      question: What is the fourth root of 1296?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c38wkAVA-dsh"
      },
      "source": [
        "And provide a prompt:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CQ8st-b92-HL",
        "outputId": "b5bc9cc9-e543-42c0-ba01-22a6090827f6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overwriting mathprompt.txt\n"
          ]
        }
      ],
      "source": [
        "%%writefile mathprompt.txt\n",
        "Think carefully and answer this math problem: {{question}}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dfauBPgj-fIe"
      },
      "source": [
        "Run the eval to produce a comparison:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IAznI7r-3F1D",
        "outputId": "ce1da4dc-888d-4a3f-cf09-b20e4cfa0e84"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "\u001b[90m┌────────────────────────────────────────\u001b[39m\u001b[90m┬────────────────────────────────────────\u001b[39m\u001b[90m┬────────────────────────────────────────┐\u001b[39m\n",
            "\u001b[90m│\u001b[39m\u001b[1m\u001b[34m question                               \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m [openai:gpt-4-0613] Think carefully an \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m [exec:python langchain_example.py] Thi \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                                        \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m d answer this math problem: {{question \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m nk carefully and answer this math prob \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m\u001b[1m\u001b[34m                                        \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m }}                                     \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\u001b[1m\u001b[34m lem: {{question}}                      \u001b[39m\u001b[22m\u001b[90m│\u001b[39m\n",
            "\u001b[90m├────────────────────────────────────────\u001b[39m\u001b[90m┼────────────────────────────────────────\u001b[39m\u001b[90m┼────────────────────────────────────────┤\u001b[39m\n",
            "\u001b[90m│\u001b[39m What is the cube root of 389017?       \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[22m\u001b[39mThe cube roo                    \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[22m\u001b[39m> Entering n                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m t of 389017 is 73.                     \u001b[90m│\u001b[39m ew LLMMathChain chain...               \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m Think carefully and answer this math p \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m roblem: What is the cube root of 38901 \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m 7?                                     \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ```text                                \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m 389017**(1/3)                          \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ```                                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ...numexpr.evaluate(\"389017**(1/3)\").. \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m .                                      \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m Answer: 72.99999999999999              \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m > Finished chain.                      \u001b[90m│\u001b[39m\n",
            "\u001b[90m├────────────────────────────────────────\u001b[39m\u001b[90m┼────────────────────────────────────────\u001b[39m\u001b[90m┼────────────────────────────────────────┤\u001b[39m\n",
            "\u001b[90m│\u001b[39m If you have 101101 in binary, what num \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[22m\u001b[39mThe number 1                    \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[22m\u001b[39m> Entering n                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m ber does it represent in base 10?      \u001b[90m│\u001b[39m 01101 in binary represents 45 in base  \u001b[90m│\u001b[39m ew LLMMathChain chain...               \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m 10.                                    \u001b[90m│\u001b[39m Think carefully and answer this math p \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m roblem: If you have 101101 in binary,  \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m what number does it represent in base  \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m 10?                                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ```text                                \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m 2**5 + 2**4 + 2**3 + 2**2 + 2**1 + 2** \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m 0                                      \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ```                                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ...numexpr.evaluate(\"2**5 + 2**4 + 2** \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m 3 + 2...                               \u001b[90m│\u001b[39m\n",
            "\u001b[90m├────────────────────────────────────────\u001b[39m\u001b[90m┼────────────────────────────────────────\u001b[39m\u001b[90m┼────────────────────────────────────────┤\u001b[39m\n",
            "\u001b[90m│\u001b[39m What is the natural logarithm (ln) of  \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[22m\u001b[39mThe natural                     \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[22m\u001b[39m> Entering n                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m 89234?                                 \u001b[90m│\u001b[39m logarithm of a number is not something \u001b[90m│\u001b[39m ew LLMMathChain chain...               \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m  that can be easily calculated without \u001b[90m│\u001b[39m Think carefully and answer this math p \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m  a calculator or a computer. However,  \u001b[90m│\u001b[39m roblem: What is the natural logarithm  \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m using a calculator, the natural logari \u001b[90m│\u001b[39m (ln) of 89234?                         \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m thm (ln) of 89234 is approximately 11. \u001b[90m│\u001b[39m ```text                                \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m 397210.                                \u001b[90m│\u001b[39m log(89234)                             \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ```                                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ...numexpr.evaluate(\"log(89234)\")...   \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m Answer: 11.399017411862108             \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m > Finished chain.                      \u001b[90m│\u001b[39m\n",
            "\u001b[90m├────────────────────────────────────────\u001b[39m\u001b[90m┼────────────────────────────────────────\u001b[39m\u001b[90m┼────────────────────────────────────────┤\u001b[39m\n",
            "\u001b[90m│\u001b[39m If a geometric series has a first term \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[22m\u001b[39mThe sum of t                    \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[22m\u001b[39m> Entering n                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m  of 3125 and a common ratio of 0.008,  \u001b[90m│\u001b[39m he first n terms of a geometric series \u001b[90m│\u001b[39m ew LLMMathChain chain...               \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m what is the sum of the first 20 terms? \u001b[90m│\u001b[39m  can be found using the formula S_n =  \u001b[90m│\u001b[39m Think carefully and answer this math p \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m a * (1 - r^n) / (1 - r), where a is th \u001b[90m│\u001b[39m roblem: If a geometric series has a fi \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m e first term, r is the common ratio, a \u001b[90m│\u001b[39m rst term of 3125 and a common ratio of \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m nd n is the number of terms.           \u001b[90m│\u001b[39m  0.008, what is the sum of the first 2 \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m Substituting the given values into the \u001b[90m│\u001b[39m 0 terms?                               \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m  formula, we get:                      \u001b[90m│\u001b[39m ```text                                \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ...                                    \u001b[90m│\u001b[39m 3125 * (1 - 0.008**20) / (1 - 0.008)   \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ```                                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ......                                 \u001b[90m│\u001b[39m\n",
            "\u001b[90m├────────────────────────────────────────\u001b[39m\u001b[90m┼────────────────────────────────────────\u001b[39m\u001b[90m┼────────────────────────────────────────┤\u001b[39m\n",
            "\u001b[90m│\u001b[39m A number in base 7 is 3526. What is th \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[22m\u001b[39mThe number i                    \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[22m\u001b[39m> Entering n                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m is number in base 10?                  \u001b[90m│\u001b[39m s 1303 in base 10.                     \u001b[90m│\u001b[39m ew LLMMathChain chain...               \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m Think carefully and answer this math p \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m roblem: A number in base 7 is 3526. Wh \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m at is this number in base 10?          \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ```text                                \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m 7**4*3 + 7**3*5 + 7**2*2 + 7**1*6      \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ```                                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ...numexpr.evaluate(\"7**4*3 + 7**3*5 + \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m  7**2*2 + 7**1*6\")...                  \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ...                                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m├────────────────────────────────────────\u001b[39m\u001b[90m┼────────────────────────────────────────\u001b[39m\u001b[90m┼────────────────────────────────────────┤\u001b[39m\n",
            "\u001b[90m│\u001b[39m If a complex number is represented as  \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[22m\u001b[39mThe magnitud                    \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[22m\u001b[39m> Entering n                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m 3 + 4i, what is its magnitude?         \u001b[90m│\u001b[39m e of a complex number is calculated us \u001b[90m│\u001b[39m ew LLMMathChain chain...               \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ing the formula √(a² + b²), where a an \u001b[90m│\u001b[39m Think carefully and answer this math p \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m d b are the real and imaginary parts o \u001b[90m│\u001b[39m roblem: If a complex number is represe \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m f the complex number, respectively.    \u001b[90m│\u001b[39m nted as 3 + 4i, what is its magnitude? \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m So, for the complex number 3 + 4i, its \u001b[90m│\u001b[39m ```text                                \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m  magnitude would be √(3² + 4²) = √(9 + \u001b[90m│\u001b[39m (3**2 + 4**2)**(1/2)                   \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m  16) = √25...                          \u001b[90m│\u001b[39m ```                                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ...numexpr.evaluate(\"(3**2 + 4**2)**(1 \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m /2)\")...                               \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m Answer: 5.0                            \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m > Fin...                               \u001b[90m│\u001b[39m\n",
            "\u001b[90m├────────────────────────────────────────\u001b[39m\u001b[90m┼────────────────────────────────────────\u001b[39m\u001b[90m┼────────────────────────────────────────┤\u001b[39m\n",
            "\u001b[90m│\u001b[39m What is the fourth root of 1296?       \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[22m\u001b[39m6                               \u001b[90m│\u001b[39m \u001b[32m\u001b[1m[PASS] \u001b[22m\u001b[39m> Entering n                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ew LLMMathChain chain...               \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m Think carefully and answer this math p \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m roblem: What is the fourth root of 129 \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m 6?                                     \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ```text                                \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m 1296**(1/4)                            \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ```                                    \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m ...numexpr.evaluate(\"1296**(1/4)\")...  \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m Answer: 6.0                            \u001b[90m│\u001b[39m\n",
            "\u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m                                        \u001b[90m│\u001b[39m > Finished chain.                      \u001b[90m│\u001b[39m\n",
            "\u001b[90m└────────────────────────────────────────\u001b[39m\u001b[90m┴────────────────────────────────────────\u001b[39m\u001b[90m┴────────────────────────────────────────┘\u001b[39m\n",
            "======================================================================\n",
            "\u001b[32m✔\u001b[39m Evaluation complete.\n",
            "\n",
            "Run \u001b[92mpromptfoo view\u001b[39m to use the local web viewer\n",
            "Run \u001b[92mpromptfoo share\u001b[39m to create a shareable URL\n",
            "======================================================================\n",
            "\u001b[32m\u001b[1mSuccesses: 14\u001b[22m\u001b[39m\n",
            "\u001b[31m\u001b[1mFailures: 0\u001b[22m\u001b[39m\n",
            "Token usage: Total 49, Prompt 34, Completion 15, Cached 448\n",
            "Done.\n"
          ]
        }
      ],
      "source": [
        "!npx promptfoo@latest eval -c mathconfig.yaml --no-progress-bar"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UdjUshgS-kg8"
      },
      "source": [
        "And view the results in the web viewer:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZOgBi_Er3rgv",
        "outputId": "3150bee8-7fa8-4fdd-eb34-d0b2790c076c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "View results: \u001b[92m\u001b[1mhttps://app.promptfoo.dev/eval/f:f447942f-42a9-46fa-b38e-3ec7bf860418\u001b[22m\u001b[39m\n"
          ]
        }
      ],
      "source": [
        "!npx promptfoo@latest share --yes"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
